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Abstract 

We prove, without recourse to the Extended Riemann Hypothesis, 
that the projection modulo p of any prefixed polynomial with inte- 
ger coefficients can be completely factored in deterministic polynomial 
time if p — 1 has a (lnp)'^(^)-smooth divisor exceeding {p — 1)5+* for 
some arbitrary small S. We also address the issue of computing roots 
modulo p in deterministic time. 

1 Introduction 

Factoring polynomials over finite fields in deterministic polynomial time is 
a long-standing open problem of computational number theory. The most 
important results obtained, though partial, are now classic. Berlekamp [2\ 
was the first to devise a general deterministic algorithm for this problem; its 
running time bound p{d\np)'-^^^\ where p is the characteristic of the finite 
field and d the degree of the polynomial to be factored, can be seen as poly- 
nomial only if p is fixed. A better and so far best time bound p2 (^dlnp)'-'^^^ 
is achieved by an algorithm of Shoup [E]. There are also algorithms with 
running time bound of the form {dl'np)'^^^\ such as the Cantor-Zassenhaus 
algorithm [4J (actually dating back to Legendre), but these use random- 
ness in an essential way. In this article we pursue an approach developed 
by von zur Gathen [6J and Ronyai [13], which consists in taking advantage 
of the multiplicative structure of p — 1. Shoup [H] refined the algorithms 
of von zur Gathen and Ronyai, improving the running time bound from 
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P+(p-l)((ilnp)<^(i) to P+(p-l)5(dlnp)'^(i), where is the largest 

prime factor of p — 1. These three algorithms are deterministic, however in 
their proofs of correctness the Extended Riemann Hypothesis must be as- 
sumed. We prove, without recourse to the ERH, that the projection modulo 
p of any prefixed polynomial with integer coefficients can be completely fac- 
tored in deterministic polynomial time if p— 1 has a (lnp)*-^'^^)-smooth divisor 
exceeding {p — 1)5+'^ for some arbitrary small 6. 

Theorem 1.1 Let f be an irreducible polynomial of degree d in 1^[X], with 
leading coefficient I, and f the polynomial o/Z[y] defined by f{Y) = l^^^f{^) 
Let h be the class number ofQ{6), where 6 is any complex root of f . Let p 
be a prime and q the least prime such that the q-smooth part S of p — 1 is 
no less than {p — 1) 2"'"'' for some (5 > 0. Then the complete factorization of 
f modulo p can be found in 06»,c,<5,e(('?^ \nq + \np) In^'*''^'*''^'^ p) deterministic 
time, where 7 is any positive number and c, e positive numbers satisfying 

Let us emphasize that the above big-O constant depends severely on 9 as 
to include the time of some necessary precomputations in the number field 
Q{9). For a thorough exposition of constructive algebraic number theory we 
refer the reader to [llj. In a sense, the idea of fixing the polynomial / rather 
than the prime p is at the opposite of Berlekamp's algorithm. Actually, it 
has been already considered by Schoof |14) in the case when / = X'^ — a 
for a an integer, and by Pila when / = — 1 for s a prime dividing 
p — 1. Both authors gave unconditional algorithms that factor the corre- 
sponding polynomial / modulo p in respectively Oa (In^"*"*^ p) and (Inp)'^^^^^ 
deterministic time. We address more generally the issue of computing roots 
modulo p in the ensuing corollary. Our proofs are mainly based upon alge- 
braic number theory, whereas Schoof's and Pila's rely on heavy machinery 
of algebraic geometry and this is of independent interest. 

Corollary 1.2 Letp be a prime andq^ the least prime such that the q-smooth 
part of p — 1 is no less than {p — 1)2+^ for some (5 > 0. Let n be a posi- 
tive integer. Suppose that the integer a is an n-th power residue modulo p. 
Then all the n-th roots of a modulo p can be computed in Oa,n,c,5,eiiQ^ lng-|- 
In J?) In^^'^^'^^ deterministic time, where 7 is any positive number, c, e pos- 
itive numbers satisfying ^ + e < 26, and H the largest integer among the 
class numbers of the fields Q{0), 6 running through the complex n-th roots 
of a. 
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In the cases covered by Schoof or Pila and such that the corresponding 
integer H is equal to one the stated running time bound is shghtly better 
for a sparse, but infinite set of primes p. It is worth noting that our technique 
combined with the result of Pila and an observation of Tsz-Wo Sze ^17j leads 
for n an odd prime to a stronger theorem than corollary 11.21 (see last remark 
of section [5]) . 

2 Notation 

In all that follows, / is a fixed, irreducible polynomial of degree d in 
with leading coefficient I, and / the corresponding monic, irreducible poly- 
nomial l^~^f{^) of degree d in ^[Y]. The number field K is the extension 
of Q by a complex root 8 of the polynomial /. The class number of K is h, 
its ring of integers - Ok- A fixed, integral basis uj = (ui, . . . ,0;^) of Ok, as 
well as a fixed, finite set U of generators of the group of units are given. 
The ideals of Ok that we consider are always supposed to be nonzero. The 
norm N{I) of an ideal in Ok is the cardinality of Ok /I- We let il)K{x,y), 
respectively iljK{x,y), be the number of ideals, respectively principal ideals, 
of Ok with norm at most x that can be written as a product of prime ideals, 
respectively principal ideals, of Ok with norm at most y. 
The letter p denotes an odd prime number. For g E Zp[y], by Rg we mean 
the quotient ring 'Lp\Y]/{g) and by i?* its multiplicative group. If the com- 
mutative group G is a direct sum of two subgroups Gi , G2 , and Q \s a, subset 
of G then the symbol {Q)g-, respectively (^)gi j stands for the subgroup of G 
generated by Q, respectively the subgroup of G/G2 generated by the cosets 
9G2, g ^Q- 

3 Auxiliary results 

We will seek to compute the factorization of / modulo p; it gives the factor- 
ization of / by a change of variable whenever p does not divide I. In general, 
we can assume that the prime p exceeds any given constant, since for small 
p factoring in Zp[X] is "easy". 

The algorithm of Fellows and Koblitz [5j for proving the primality of an 
integer n or also the deterministic version of Pollard's p — 1 method [18] 
for factoring n, perform certain operations on a "small" subset B oi'L that 
generates modulo n a "large" multiplicative semigroup S. In fact, B can be 
chosen there as the set of prime numbers not exceeding In^ n; the semigroup 
S has then at least 'tl^in, In'^ n) elements, where is the de Bruijn function. 
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Here similarly, to factor / modulo p we will construct a small subset of 
Zp[y]/(/) generating a large multiplicative semigroup. By the following 
lemma, the latter task amounts to exhibiting a suitable subset of Ok^ at 
least if p is sufficiently large. 

Lemma 3.1 Assume that p does not divide the index [Ok '■ Then 
I— > y induces an isomorphism k : Ok/{p) '^p[^]/{f)- 

Unlike Z however, Ok is not a unique factorization domain (unless h = \). 
It is still a Dedekind domain and just as V measures the smoothness of 
integers in Z, the function tpK measures the smoothness of ideals in Ok- 
The next theorem generalizes in this sense a result of Canfield et al. [3] . 



Theorem 3.2 (Moree, Stewart) There is an effective, positive constant 

In a: 
\ny 
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ci = ci{K) such that for x,y > 1 and u := > 3 we have 



ipK{x,y) > xexp 



I In In n — 1 / In In n 

-u < In(nlnti) — 1 H h ci — 

Inn V mu 



The generator of a principal ideal in Ok is defined up to a multiplicative unit 
of Ok, so working with principal ideals, rather than general ones, is pretty 
much like working with integers. That is why we give via ■i/'K a lower bound 
for the function ipK counting the number of "smooth" principal ideals. 

Lemma 3.3 There is an effective, positive constant C2 = C2{K) such that 
i^K{x,y)> \i^K{c2X,yT^)fory>C2^. 

Proof. Let /i, . . . , J/i be a set of representatives for the class group of 
K whose norms are bounded above by the Minkowski bound Mk- We will 
prove that the lemma holds with C2 = ^k^- Define ip'j^{x, y) as the number 
of principal ideals of Ok with norm at most x that split as a product of 
prime ideals of Ok with norm at most y. 

Let J be an ideal counted by 'iljK{M^^x,yh). There exists a k, 1 < k < h, 
such that J/fc is principal. Suppose that yh > Mk, i.e. y > M^. Then J/^ 
is counted by ip'^{x,yh). Moreover, any ideal counted by il)'K{x,yh) can be 
written in at most h ways as Jlk, where J is counted by ^KiM^^x, yh ) and 
1 < k < h. Consequently, j^ipKiM^^x,yh) < ilj'j^{x,yh). 
Assume that the principal ideal / of Ok is a product of m prime ideals of 
Ok with norm at most y^. It is easy to show by induction on m that / 
is a product of principal ideals of Ok with norm at most y. Just use the 
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fact that every product of at least h ideals of Ok contains a principal factor. 
Therefore any ideal counted by ip'j^{x, yh) is also counted by iPk{x, y), hence 

i^'A^^y'^) < 4^K{x,y). □ 

It becomes apparent that if we let B be the union olU and a set A containing 
pairwise non-associate integers with small norm then it should generate 
modulo p a relatively large multiplicative semigroup S. Nevertheless, three 
problems arise. Is S indeed large though reduction modulo p? Can the 
suitable set A be small and easy to find? The ensuing theorem helps to 
answer these questions positively. 

Theorem 3.4 (Fincke, Pohst) There is an effective, positive constant C3 = 
c^{K,uj) such that for any rj € Ok \ {0} there exists fj £ Ok generat- 
ing the same ideal as rj and whose coordinates ai in the basis uj satisfy 
\ai\ < csNUt]))!. 

Proof. Combine the equations (3.5b), chapter 5, and (4.3f), chapter 6 of 

m- □ 

We now summarize rigorously the above informal discussion. Actually, we 
show more: for any g dividing / modulo p, a set Bg derived from B generates 
a large multiplicative semigroup in Rg. 

Lemma 3.5 Suppose that the polynomial g of degree d' divides f modulo 
p. Let p and k be as in lemma WlX Let tt and Hg be the projections Ok 
Ok/{p) and 'Lp\Y]/{f) — > 'Lp\Y]/{g) respectively. Fix c > and define 

ch 

A = {aiuji + . . . + OdUJd : Oj G Z, |aj| < C3 In" 1 < i < d}, S = {v ■ ai ■ 
. ..-am ■■ V e 0*j^, m G N, Oi e A, 1 <i < m}. Then #7rgK7r(5) > p'^^"^ 
for any €>0 and p >po, Po = Po(c, ci, C2, C3, e). 

Proof. Let T = S (1 {aiwi + . . . + adUJd : Oj G Z, \ai\ < |, 1 < 
i < d}. It is sufficient to prove that the desired inequality holds with S 
replaced by T. We invoke theorem [3^ to get #T > '0i^((2|^)'^, In^'^'p). 
By lemma 13.31 if p is large enough the latter expression is no less than 
j^iPk{c2{2^)'^A^'^ p)- This in turn is greater than p'^~~~'^ for any e > 

and sufficiently large p, by theorem 13.21 Thus #T > p'^~c~'^ if p exceeds 
some constant po depending upon c,ci, 02,03 and e. Assume that it does. 
As p > 2, we have #7r(T) = if^T. Furthermore, k is an isomorphism, 
hence ^ktt(T) = ^it{T). Finally, vr^ is a surjective homomorphism, so 
the preimage under -Kg of any element of 'Ijp[Y]/{g) has #ker7rg = p'^~'^ 

elements. It follows that p"^'"^' ■ #7rgK7r(T) > #K7r(r) = #T > p'^'i'". 
Therefore #7rgK7r(T) > /'-c-^ □ 
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Assume that g is a product of at least two distinct, degree e irreducible 
factors of / modulo p. Either the set Bg mentioned above is not contained 

in R* U {0}, or Q := {b p-^ : b £ Bg \ {0}} generates a large subgroup 
of {a G Rg : a^~^ = 1} thus should not be cyclic. The latter case is 
dealt with an extension of the Pohlig-Hellman algorithm [lOj for computing 
discrete logarithms. 

Theorem 3.6 Let g be a polynomial of degree d! in 'L.p\Y\ and G the group 
{a e R*g : aP-^ = 1}. Write G = d ® G2, (#Gi,#G2) = 1. Suppose 
that we are given g, #Gi and a subset G of G such that {G)gi is not cyclic. 
Then we can find a nontrivial divisor of g in 0^i[^Q ■ [q2 Ing + lnp) ln^~'"^p) 
deterministic time, where q is largest prime factor o/#Gi and 7 any positive 
number. 

Proof. The deterministic Pollard-Strassen ^12j algorithm can be used to 
find the complete factorization of the g-smooth part oi p — 1 in the stated 
time. The rest of the proof is based on obvious modifications of the proofs 
of corollary 4.4, theorem 6.6 and on remark 4.3 from [TSj. □ 



4 Proof of theorem 11.11 

( -dS-l \ 
I, [Ok '■ '^[S]],Po, (l — Po~^) I , where po is the constant 

from lemma 13. 5( then we can find efficiently the complete factorization of 
/ using the deterministic Berlekamp algorithm for example. Now assume 
that the reverse inequality holds. We first compute the squarefree, distinct- 
degree factorization of / modulo p, that is the products ig, e G N, of all 
distinct, degree e irreducible divisors of / modulo p. Fix e; the complete 
factorization of t^ will be found by using the following inductive procedure. 
Let 3 be a factor of te of degree d' , say d' = ke. Suppose that k > 2. 
We show below how to split g nontrivially. Keep the notation of lemma 
13. 5[ Define Bg = -KgH-KiJA U A). We can assume that Bg C R* Li {0}; in 
the contrary case {b,g) is a nontrivial divisor of g for some b (z Bg. Let 
F = Bg \ {0}. With G as in theorem 13.61 let Q be the image of !F under the 
homomorphism a : R* ^ G raising every element to the power p^~^ . Write 
G as Gi ® G2 with #Gi = S'' - this condition uniquely determines Gi and 
G2 . We apply the algorithm from theorem 13.61 to check whether the group 
{Q)gi is cyclic. Suppose that it is, for otherwise we would obtain a nontrivial 
factor of g. Then the order of {G)gi divides p — 1. We will estimate this 
order from below to obtain a contradiction. We have #(^)gi = # {gfc^G2 ' 
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The kernel of a has (frr) elements, hence #{Q)g > (^) ' 

We appeal to lemma [331 to deduce that #{J^)r* > p'^^~^^ — 1. Since 

S>{p- it follows that #((^)g n G2) < #G2 < {p - l)^'''^. There- 

fore #{G)gi > {p~ 1) 2"^^'' • ^-j^^zrj^- The right hand side of this inequality 
is easily seen to be no less than p — 1, which gives the desired contradiction. 
This means that a nontrivial divisor of g had to be found at some stage. 
Once we have completely factored / in Zp[y], we get the complete factor- 
ization of / in 7jp[X] by the change of variable Y = IX. 
Obviously, the most time-consuming part of the described procedure is test- 
ing whether {G)gi is cyclic. The stated running time follows from theorem 
ESI as #g = Od{\n'^p). 

5 Concluding remarks 

Theorem 13.61 can be applied to get a short proof of Shoup's result from 
[16j , which states that under the ERH there is an algorithm that completely 
factors any degree d polynomial / in Zp[X] in P^{p — 1)2 (^dlnp)^^^^ de- 
terministic time. It suffices to iterate the following procedure. Let g he a 
reducible factor of /. Adopt the notation of theorem 13. 61 As in von zur 
Gathen's algorithm, either we find directly a nontrivial divisor of 5, or com- 
pute an element a of G \ Zp whose order is the power of some prime s (cf. 
[U). Then we find an s-th power nonresidue b modulo p. Finally we use the 
algorithm from theorem 13.61 with Gi = G and G = {a, b} to find a nontrivial 
divisor of g. All the required steps can be done in the stated time. 
Instead of completely factoring we could be simply interested in splitting 
the polynomial / modulo p. To this end, the running time bound obtained 
in theorem 11.11 could be in some cases largely improved. For example, if the 
degree of / is odd then it would be sufficient to take the integer q therein as 
the least prime such that the (7-smooth part of p — 1 is no less than {p—l)3~^^ 
for some S > 0. 

As another example, consider the polynomial / = — a, where s is an 
odd prime number and the integer a is not an s-th power. Suppose that 
/ splits modulo p into distinct linear factors, or equivalently a is an s-th 
power residue modulo p and s divides p — 1. In order to split / modulo p 
within the time bound of theorem 11.11 it is then enough to choose q as the 
least prime such that the g-smooth part of p — 1 is no less than {p — 1)"^'' 
for some S > 0. The point is that a nontrivial factor of / modulo p leads 
to an s-th root of a modulo p (cf. [T7]). The remaining s-th roots of a and 



7 



hence the complete factorization of / modulo p can be found by computing 
a primitive s-th root of unity modulo p. This in turn can be done using 
Pila's algorithm [9]. 
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